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ABSTRACT 

During the 2002-2003 academic year, six roundtables were held 
to explore uses of the data from the National Survey of Student Engagement 
(NSSE) for improvement of student learning. This paper describes the 
roundtable discussion held in Kentucky in December 2002. This roundtable 
focused on ways the NSSE can contribute to state-level efforts on 
accountability and institutional improvement. Participants included 
individuals with state-level responsibilities for higher education and 
representatives of three universities. Kentucky has already taken steps to 
combine attention to accountability with efforts to support institutional 
improvement. In recent years, all colleges and universities in the state have 
engaged in substantial activity to demonstrate and increase their educational 
effectiveness. Statewide surveys and workshops have been conducted, and the 
state has set specific goals and timelines for institutional action. In its 
review of Kentucky's experience, the roundtable focused on: (1) state-level 

uses of NSSE data; (2) institutional perspectives on uses of NSSE data; (3) 
observations of greater uses of NSSE data; and (4) NSSE data as a new source 
for- communicating with important audiences. (Author/SLD) 
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State-Level Uses of NSSE Data: Observations from Kentucky 

BY Elaine El-Khawas 



During the 2002-03 academic year, AAHE and NSSE are conducting six roundtables to 
explore uses of NSSE data for improvement of student learning, A roundtable discus- 
sion held in Kentucky on December 3, 2002, focused on ways that NSSE can contribute 
to state-level efforts on accountability and institutional improvement, Kentucky’s 
Council on Postsecondary Education (CPE) co-sponsored and arranged the meeting. 
Participants included persons with state-level responsibilities for higher education and 
representatives of three universities. 

Kentucky already has taken steps to combine attention to accountability with efforts to 
support institutional improvement. In recent years, all colleges and universities in the 
state have engaged in substantial activity to demonstrate and increase their educational 
effectiveness. Statewide surveys and workshops have been held, and the state has set 
specific goals and timelines for institutional action. 

In its review of Kentucky’s experience, the roundtable focused on: 
v State-level uses of NSSE data, 
v Institutional perspectives on uses of NSSE data, 
v Observations for greater use of NSSE data, and 

> NSSE data as a new source for communicating with important audiences. 



State-level Uses of NSSE Data 

Using a consortia model, Kentucky has made extensive use of NSSE as a basis for meeting state goals for 
improvement. This approach offers two important benefits: the sponsoring body (i,e., the Council on 

Postsecondary Education) obtains a full set of study results, by institution, and the consortium can add up to 20 
questions to the survey form. All Kentucky institutions participated in the 2001 NSSE survey and will participate 
again in 2003. 

As one participant noted, “An important benefit of NSSE data is that they help put Kentucky’s progress and 
efforts into perspective.” Comparisons to national data - whether showing that Kentucky scores lower or higher 
than others — help in understanding the state’s current status and what needs to improve. NSSE results have 
pointed to “lower than predicted” performance on some aspects of the undergraduate student experience, for 
example. 

Kentucky has found NSSE data helpful in several ways: 

■ As benchmarks for assessing progress toward state goals — To promote accountability and 
institutional improvement, Kentucky has developed a far-reaching set of state-level goals for 
postsecondary reform. Three of these goals focus on college preparation, enrollment growth, and student 
progress toward degrees. Two other goals address broad state priorities: that postsecondary institutions 
should prepare graduates for life and work and that Kentucky’s communities and economy should benefit as 
well. The Council on Postsecondary Education (CPE) has developed a Key Indicators project to 
monitor progress on this reform initiative. 

NSSE surveys offer a means to monitor improvement on several of the state’s goals. NSSE results from 
2001 offered valuable information, for example, on how well Kentucky institutions are promoting civic 
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engagement, which is relevant to the state’s goal of preparing students for life and work. As one participant 
commented, “Such hard data on civic engagement can help demonstrate to university stakeholders - families, 
community leaders, the legislature, the business community - that universities help to improve the quality of life 
and work in Kentucky.” 

■ As a source of insight about student experiences — CPE organizes an annual statewide faculty development 
conference to promote institutional improvement. Participants (about 400 to 500 faculty across all campuses) 
hear about good practices, both from invited speakers and through presentations by Kentucky 
institutions. NSSE data have helped to identify topics for attention at these conferences, and, because they 
cover each topic in sufficient depth, they allow discussion of what’s working and not working for students. 

■ As confirmation of what is shown by other studies — Every state has a variety of statistics and studies on its 
postsecondary institutions, but findings from many home-grown studies can be inconclusive. Related data 
found in the NSSE surveys provide a larger context and a stronger basis for considering needed changes. For 
example, a recent state-level survey of alumni suggested that advising should be improved. The NSSE study, 
which had similar findings, confirmed this view. As a result, CPE is now planning a statewide conference on 
advising, which will address issues in advising first-year students and students in their majors. 

Using the 2001 survey as a baseline, CPE expects all institutions to show progress on the NSSE survey by 2003. 

The goal is to have all institutions attain results that are above their predicted levels on all five NSSE benchmarks. 



Institutional Perspectives on Uses of NSSE Data 

University representatives at the meeting readily cited examples of NSSE’s use as a diagnostic device for 
identifying weaknesses in current practices and as a way to monitor whether improvement efforts are making a 
difference. 

Identifying Problem Areas 

NSSE data offer detailed information about various aspects of the student experience. Examples cited include im- 
proving advising, making general education a more meaningful experience, promoting community service, 
demonstrating that teaching is valued, improving support services for graduate students, and developing broad 
standards for graduate programs. The provost at one university established a task force with three working groups that 
focus, respectively, on undergraduate students, graduate students and programs, and the institutional environment. 
Where NSSE results identify problems in these three priority areas, recommendations about needed effort are made to 
the appropriate group. 

Each university has focused on different areas for improvement. Based on NSSE results, one university created a 
task force on the first year experience for students. At another university, review of the NSSE data pointed to the need 
for systematic attention to seniors. Its Assessment Committee is currendy talking about ways that capstones or other 
synthesis experiences might be developed. Another participant noted that her university has a student success coun- 
cil — a cross-functional grouping from diverse program areas — that has found NSSE data useful in identifying poten- 
tial solutions to problems. As one result, the institution’s teaching and learning center is drawing on NSSE data to plan 
workshops to improve collaborative learning. NSSE data have helped another university become aware of student co n- 
cerns that bureaucratic structures work against student needs. Data are helping internal groups find ways to streamline 
administrative procedures and make sure that key staff have the information they need to assist students. 

Monitoring programs and progress 

Kentucky universities have used NSSE results to assess their current programs and also to monitor whether 
improvement initiatives are having good results. NSSE data were especially instructive with respect to one university s 
recent initiatives for first-year students. NSSE scores among first- year students improved significantly on the 
benchmark for academic challenge over three years. These data provide evidence that several institutional 
efforts - new orientation days with increased academic content, a new advising center, revisions and improvements in 
University 101, new initiatives with the teaching and learning center — are making a difference. 

One university used NSSE results to evaluate responses of students who were part of the Freshman Discovery 
seminars. At another university, NSSE data were “unpacked” by department or office so that they could be used in 
annual assessment reports. NSSE results on advising, for example, were given to the undergraduate advising office; 
NSSE results on diversity were provided to the office on minority affairs. At one university, the new president has 
called for measures to gauge the University’s goal of being among the top universities in the United States. Several 
NSSE benchmarks will serve as measures for this important presidential initiative. 
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Observations: Toward greater Use of NSSE data 

Roundtable participants offered several suggestions for how NSSE and other data can contribute to efforts toward 
postsecondary improvement. 

■ A major strength of the NSSE approach is that it offers institutional comparisons. For those items on 
which one university has lower than desired scores, it can be instructive to look at practices of schools that have 
higher scores. Comparisons with other institutions greatly strengthen the pattern, even if the comparison is only 
with eight to ten others. NSSE allows data to be reported by institution if all institutions agree, but individual 
states need to work out specific understandings on disclosure in order to make the best use of the data. 

■ To look at educational progress for a state, it’s critical to have benchmarks. “Scores for Kentucky only 
take us so far,” one participant said. “We need to compare these scores to the national picture and against other 
states. When we see a change in Kentucky’s scores, we n eed to know whether the national trend is comparable: 
if we had a decline in some area, did others show a decline and the same amount of decline?” 

■ Multiple measures and surveys complement each other and help confirm that action is needed. 
NSSE’s focus on specific student behaviors and experiences is a good complement to other, mosdy factual, 
cam pus -derived data on enrollment patterns and also to the small-scale studies that states and campuses 
regularly conduct (for example, on graduating seniors). 

■ Emphasis should be on steps being taken for improvement. It’s important to find momentum and not to 
expect too much change right away. For many efforts, three or more years may be needed to show 
measurable progress in student experiences. Disappointing NSSE results can signal areas for change if results 
are interpreted with an eye toward improvement. 

■ Comparisons of different groups of students can lead to new insights. What are the patterns of behavior 
for students with high scores on certain experiences, for example, compared to students with low scores on the 
same experiences? 

■ NSSE helps examine the student experience from first-year to senior year. Where scores for seniors seem 
to be lower than those for freshmen, for example, more research might be useful Do scores decrease during 
senior year, or have they dropped among junior-year students too? Depending on what patterns are found, 
different institutional interventions would be appropriate. 

■ NSSE data can offer new perspective on the senior year. One participant asked, “Are we correct in assum- 
ing that the senior year is primarily a fourth year of study, or is it better understood as a transition year in which 
attention has shifted to post-university decisions?” If so, universities may better serve seniors by offering better 
preparation for post-university life. 

■ NSSE data provide a basis for new insights into problems of low persistence. Profiles might be 
developed of those freshmen that are “at-risk” of dropping out, for example. The instructors of first-year 
students should have this information to guide their teaching. Also, NSSE data on transfer students offer 
additional information on their transition experiences. For some universities, a substantial proportion of 
graduates have transferred into the institution. 

■ NSSE data help with improvement efforts targeted to various programs and academic departments. 

NSSE results lend themselves to assessment of general education and also to assessments of major areas of 
study. NSSE results and their implications should be discussed at the department level to encourage faculty 
interest and discussion. Many other ampus offices — for example, those arranging internships and service 
learning, career planning, residence life, and other areas — could benefit from the use of NSSE data. Insights 
from NSSE may help identify new models for effective student advising, especially how it could be structured 
for beginning students. 

NSSE Data: ANew Source for Communicating with Important Audiences 

A number of participants pointed out that the NSSE data have been welcome as a new source of information. 
Because it addresses different questions, compared to other sources, NSSE illustrates new aspects of what universities 
do. The NSSE data have been useful in responding to questions from various constituencies about the student 
experience. Some observations include: 

■ NSSE data help create a common language for discussing improvement. They help identify broad and 
important issues that need to be part of institutional and policy conversations. One participant remarked, 
“NSSE has provided specific examples and ideas for discussing issues of academic challenge and student 
engagement, and also for discussing ways to enrich the academic environment.” Another participant added, 
“NSSE results help give direction to improvement efforts, because survey responses not only point to a 
problem but offer suggestions for how to improve things.” 
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■ NSSE results relevant to institutional improvement should also be shared internally. One university 
reported that a six-page electronic newsletter, summarizing results on the five NSSE benchmarks, is sent to 
deans, other senior administrators, and department chairs. Timely placement of NSSE data on the institution’s 
website has been helpful, for example, in informing several administrators who are new to leadership positions 
at the university. 

■ NSSE data should be used to communicate directly to the public. Community events could be scheduled 
around discussion of the NSSE results; informal forums could be scheduled with legislators, especially to 
discuss areas showing progress. The NSSE results could be very useful in building public support for the 
universities, because the results show that the universities are involved in helping to make Kentucky a better 
place to live. 

■ NSSE data should be useful to the media. This may call for working directly with reporters to facilitate their 
use of the data. Focusing on segments of the data, or certain topics, may be useful. 

■ Providing information back to high schools should be a priority. The NSSE data offer a good resource 
for conveying to high schools what expectations are at the university level. Such information could be directed 
to school superintendents, principals, teachers, and parents. High school students would also benefit from 
having specific information on what to expect of the college experience. 

Final Thoughts 

NSSE is distinctive in that it focuses on behaviors and experiences known to be related to student success in 
educational and personal development. The day’s discussion helped to identify what is needed to promote wide use of 
this important data for improvement of student success and for accounting to multiple publics at the local and state 
level about student learning 
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